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Abstract 

In this paper, we present the design and development of an undersea camera system. The goal of our system is 
to provide a 3D model of the undersea habitat in a long-term continuous manner. The most important feature of our 
system is the use of multiple cameras and multiple projectors, which is able to provide accurate 3D models with 
an accuracy of a millimeter. By introducing projectors in our system, we can use many different structured light 
methods for different tasks. There are two main advantages comparing our system with using ROVs or AUVs. First, 
our system can provide continuous monitoring of the undersea habitat. Second, our system has a low hardware 
cost. Comparing to existing deployed camera systems, the advantage of our system is that it can provide accurate 
3D models and provides opportunities for future development of innovative algorithms for undersea research. 

Index Terms 

Undersea camera system. Structured light, 3D model 


I. Introduction 


T he study of underwater species and their habitat has been a popular research topic in many areas 
of science and engineering for many decades. Researchers from different disciplines study them 
for different purposes. For example, marine biologists try to understand the marine environment which 
helps to support life on earth [[T]|. Studying underwater species helps them to understand the effects of 
climate change, pollution, and invasive species. 

A typical technique for undersea monitoring uses either an Autonomous Underwater Vehicle (AUV) 
[|3l or a Remotely Operated Vehicle (ROV) [|2l to explore the underwater habitat. There are several 
disadvantages of using this technique. First, AUVs or ROVs cannot be permanently deployed undersea. 


X. Chen, S. Sutphen, and Y-H. Yang are with the Department of Computing Science, University of Alberta, Edmonton, AB, Canada, 
e-mail: {xida, ssutphen, herberty}@ualberta.ca. 

P. Macoun is with University of Victoria, Victoria, BC, Canada, email: pmacoun@uvic.ca. 


2 


Instead, it usually goes underwater onee every 4 to 6 months. Therefore, the eolleeted data ean only 
be eompared with that eolleeted several months ago, whieh may not be useful in some studies. Seeond, 
temporal data eannot be obtained for the same loeation beeause of the laek of landmarks undersea and 
the aeeuraey of GPS. Third, the eost of using AUVs or ROVs is very high. In partieular, a large ship is 
usually required to earry the AUVs or ROVs out to the sites. The entire erew of the ship is also required, 
and several biologists are typieally onsite to eolleet data. Finally, the expedition is at the merey of the 
weather. 

In this paper, we deseribe a system that is deployed permanently undersea. The goal of this system is to 
perform long-term monitoring of the underwater habitat. Comparing to using AUVs or ROVs, the eost of 
our system is signifieantly lower. Moreover, our system is able to provide information of the underwater 
habitat on demand or on an hourly or daily basis. Additionally, we have the option of streaming data in 
real time. There are three ehallenges of developing sueh a system. 

O We need to provide aeeurate 3D models of the undersea habitat from 2D images. In order to aehieve 
high aeouraey 3D reeonstruetion, an appropriate method has to be seleeted. 

@ Beeause of the above requirement, the hardware seleetion is another major ehallenge. 

© Sinee our system monitors the undersea habitat regularly, the amount of data is huge and the storage 
of it is another ehallenge. 


II. Literature Review 

Using ROVs or AUVs [IH, [|5l to explore the oeean is still a popular approaeh nowadays. They are 
useful in many ways, mostly beeause of human safety and the ability to eonduet eontinuous operations. 
In the intense underwater pressure of the marine environment, ROVs or AUVs eliminate the risk of 
aeeidents to the oeeupants of manned submersibles. The failure of pressure seals or of the life-support 
system that provides oxygen and removes earbon dioxide is not a eoneem in ROVs or AUVs. Moreover, 
ROVs ean operate in a eontinuous manner. In eontrast, due to safety eoneerns, shipboard reeovery of 
these submersibles ean often restriet operations to daylight hours. Despite of their advantages, there are 
still limitations of using ROVs or AUVs to explore the oeean. First, ROVs or AUVs are usually deployed 
undersea every 4 to 6 months. During eaeh trip, the digital eamera(s) and video eamera(s) mounted on 
them are used to eolleet as mueh data as possible. However, one fundamental issue is that the data is 
only available every 4 to 6 months and there is no data in between. Furthermore, it is diffieult if not 
impossible to go baek to exaetly the same loeation beeause of the laek of landmarks and the aeeuraey of 
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the GPS. In this case, the data may not be very helpful to biologists or ecologists. For example, in order to 
study how the pollution impact the undersea habitat, the image or video data may need to be continuous. 
Furthermore, the cost of each trip is very high. In particular, in addition to the time for planning, a typical 
ROV trip takes 3-7 days to explore different observation sites. A large ship and its entire crew are needed 
to carry the ROV to the observation site. Additionally, a group of scientists has to be onsite to navigate 
the ROV in order to collect data at the location that would be useful to them. Last but not least, the trip 
can also be impacted by weather. When the undersea turbulence is too strong due to bad weather, the 
ROV will be very hard to control. Careless control of the ROVs or AUVs could damage both the vehicle 
and the undersea habitat. 

Due to the above limitations of ROVs and AUVs, there are systems being developed to long-term 
surveillance purpose. A camera surveillance system is presented in |0 to monitor the fish populations. 
It focuses on the development of software to recognize and track fish from a very large database of 
videos. Ocean Networks Canada [|71|, [|9l has deployed many undersea systems along the western coast of 
Canada. Some systems are deployed at over 2000m deep, while others are at about 100m deep. The entire 
Ocean Networks Canada consists of NEPTUNE and VENUS cabled observatories, where NEPTUNE is 
an earthquake and tsunami research lab and VENUS an underwater landslide research lab. The collected 
data are shared with many researchers for data visualization and analysis. The data is potentially useful for 
many applications, such as studying ocean/climate change, ocean acidification, recognizing and mitigating 
natural hazards, non-renewable and renewable natural resources. With such a huge observation network, 
different functionalities can be achieved. Consider NEPTUNE as an example. It can be used to determine 
the presence of fish sounds by examining the deep sea acoustic recordings [jH. It is also used to study 
faunal grouping lITOll . The study shows that distinct seasonal faunal groupings are observed, together with 
summer and winter trends in temperature, salinity and current patterns. We can see that a long-term 
surveillance system can achieve much more than that from discrete ROV or AUV trips. 

Motivated by the NEPTUNE and VENUS systems, we design and implement a camera system that 
is currently deployed undersea to monitor the habitat. There are several unique features of our system. 
Eirst, our system operates in a continuous manner, captures images on demand or on predefined intervals 
and provides 3D model of the habitat. Second, our system is designed to be easily extended for future 
development. East but not least, the cost of our system is low. The total hardware cost is less than $140,000 
(CAD). 
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III. System Overview 

The goal of our system is to provide a 3D model of the undersea habitat from 2D images. To aehieve 
that, our system needs an image eapture module and a eontrol module. The image eapture module is in 
eharge of taking images from different views while the control module is in charge of streaming and 
collecting data from each camera. 

A. Image Capture Module 

The most important feature of our system is the introduction of multiple projectors and multiple cameras. 
We apply a structured light method to build the 3D model from 2D images. A structured light method 
is applied because of its high accuracy, which can typically limit the error to within a millimeter. The 
method requires at least one projector and one camera. We use multiple camera and multiple projectors 
so that the accuracy is even higher, and the observed area is larger. Using projectors in our system is a 
critical design for future extension. That is, projectors can be used to project different structured light 
patterns for accurate 3D object reconstruction. There are many structured light methods that can be used 
in our system [ITTIl . Some of the methods can be used to capture dynamic events, while others can achieve 
extremely high accuracy. In other words, our system is a framework that can be used to serve multiple 
purposes. 



Fig. 1: Diagram of the image capture module. 


Fig. shows the diagram of our image capture module. There are 8 cameras and 3 projectors arranged 
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in a full circle, and all the deviees are looking at a eornmon point. The deviees are arranged so that the 
system ean observe as mueh information of the habitat as possible. Eaeh eamera used in our system is a 
Point Grey Flea3 GigE eolor eamera with 5M pixels, where the data ean be streamed through the Ethernet. 
This eamera is seleeted beeause it ean be easily eontrolled through the Ethernet and the image buffer 
ean be transferred quiekly. Sinee eaeh deviee is plaeed in a watertight housing, ideally the deviee should 
generate no or very low heat. It is the main reason that we seleet the TI Eighterafter as our projeetor, 
whieh is a pieo data projeetor that ean operate without a fan. Another reason is that TI Eighterafter ean 
be eontrolled via the Ethernet for projeeting struetured light patterns, whieh is eonsistent with the eamera. 
It ean potentially aehieve a very high frame rate at over 1000 fps. 


B. Control Module 
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Eig. 2: Diagram of the eontrol module. 


Eig. shows the diagram of our eontrol module. There are two PCs for redundaney in eharge of 
eontrolling the image eapture module, where one of them is a main operating PC and the other one serves 
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as a backup in case of hardware failure. In partieular, the PC is running the software that eontrols the 
TI Lighterafter to projeet struetured light patterns and the eameras to eapture images simultaneously, as 
well as eolleeting buffered data from the eameras. In our design, the PC is earefully seleeted to satisfy 
several requirements. First of all, the PC needs to eommunieate with the image eapture module through 
the Ethernet. It also needs to eommunieate with the server in order to upload the data. As well, the PC 
should be fast. In this ease, we seleet a PC that has at least two Ethernet ports, one for eaeh of the above 
funetions so that the two separate eommunieation ehannels will not interfere with eaeh other. Another 
important requirement of the PC is that it must run without a fan. The eooling for the hardware sueh as 
the CPU should be air eooled and very effieient. As a result, the Intense PC liT^ is seleeted beeause it fits 
all of our requirements. It is a fanless PC with an Intel i7 CPU, 8GB memory, two 1Gb Ethernet ports, 
and 500GB of solid state hard drive. In the diagram, we ean see that one of the Ethernet ports of the PC 
(Eth-2) is eonneeted to two 8-port Ethernet switeh, whieh eonneets 8 eameras and 3 projeetors together. 
Another Ethernet port (Eth-I) is eonneeted to a 5-port switeh whieh eonneets to an on-shore eomputer 
as well as to a Web-I power relay board [12]. Using the Web-I relay board is another unique design of 
our system. In partieular, the board is software programmable and has 16 ports aeting as power switehes 
for the 8 eameras, 3 projeetors, 2 PCs and 2 8-port switehes. With this design, we ean use the on-shore 
eomputer to turn on/off any deviee in our system through a program. Moreover, onee one of the PCs is 
turned on, we ean also use programs to turn on/off any deviee from the undersea PC, whieh gives more 
eontrol on the image eapture module. Eor example, we may not want the projeetor to be turned on at all 
time beeause the light will attraet fish. 

C. Housing 

Eaeh eamera or projeetor is mounted in a watertight housing equipment. Moreover, the entire eontrol 
module is sealed in a watertight housing equipment. The left image in Eig. shows a eamera and its 
housing before the eamera is sealed inside. The housing equipment is tested to guarantee that it ean resist 
a pressure of 100m depth of water. The right image shows a eamera, a projeetor and their housings. 

Eig. 1^ shows the system before assembling. In partieular, the 8 eameras and 3 projeetors are put inside 
the housing and standing next to the eonneetors. The eontrol module and its housing are also shown in 
the figure. In this figure, we ean only see one side of the eontrol module. On this side, there is one PC, 
the relay board and the 5-port switeh attaehed. On the other side, the other PC, the two 8-port switehes 
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Fig. 3: Left: A camera and its housing. Right: A camera, a projector and their housings. 


are attached. With this design, the space usage is minimized. Fig. shows all the components of our 
system after they have been mounted on a supporting frame and is ready to be deployed undersea. 

IV. Software Development 

The software of our system includes two parts: system calibration and 3D reconstruction. 

A. System Calibration 

In order to obtain accurate 3D models for the underwater habitat, accurate system parameters are needed 
which are obtained by system calibration. In particular, there are two sets of parameters that are required 
for 3D reconstruction. The first includes the camera intrinsic parameters and the relative pose of cameras 
m. The second consists of the refractive interface normal and the distance from the camera center to 
the refractive interface for each camera llT4l . Both sets of parameters can be obtained when we built the 
system and before it is deployed undersea. The second set is used to accommodate for water refraction 
so that the 3D reconstruction result is more accurate. 

B. 3D Reconstruction 

We use the structured light method for 3D reconstruction of the undersea habitat, to be more specific, 
the gray code method ifTTI . After 3D reconstruction, the captured image as well as the 3D model is 
uploaded to a data server instead of storing in the PC. 




V. Results 

The prototype of our system was tested in our lab. In partieular, Fig. shows 3D reconstruetion result 
of a coral reef placed in water by using the prototype. The left image shows the result from the front 
view and right image from the top view. We can see that a lot of the surface details are captured in the 
result by using our structured light method. Fig. shows result without using structured light patterns 
[fTSll . We can see that our method is able to provide much better result than that of ifTSl . 

Our system starts capturing images once it is deployed undersea. Fig. shows images captured by two 
different cameras. Moreover, Fig. [^compares the 3D reconstruction results of using our method with that 
of using the method presented in [fTSll . 
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Fig. 5: The system before it is deployed undersea. 



Fig. 6: 3D reeonstruetion results of a eoral reef in the lab: front view (left) and top view (right). 


VI. Conclusion and Future Work 

We design and develop a eamera system for long-term monitoring of underwater habitat. We have 
demonstrated the unique and important feature of our system. Future work of our system ineludes using 
different patterns to eomplete different tasks. In our 3D reeonstruetion result (top row of Fig. |^, the top 
half of the eentered objeet is missing beeause our projeetor is ineorreetly oriented by a small amount. 
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Fig. 7: 3D reconstruction results using PMVS2 lUSll . Left: front view. Right: Top view. 



Fig. 8: Images captured by two different cameras. 


This will be corrected in the future for a full 3D reconstruction. 
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